Performance Considerations for Network Switch Fabrics on Linux Clusters

One of the most significant components in a cluster is the interconnection network between computational nodes. A majority of today’s clusters use either switched Fast Ethernet, Gigabit Ethernet, or a specialized switch fabric to connect nodes. However, the use of these specialized switch fabrics may not necessarily benefit the users, and in some cases they perform only slightly better than commodity Ethernet standards. In this paper we investigate the necessity of a high speed interconnect for Linux based clusters and the performance differences between Myrinet and Ethernet connected networks. We benchmark applications and network performance in a dedicated environment with varying workloads and processing elements. We show that for certain workloads, a high speed interconnect is not required. We also examine the increased complexity of introducing high-speed switch fabrics, the importance of their underlying device drivers, and their usefulness with real applications. Finally, we make recommendations as to which network to use given a certain workload and available computation nodes.